demand ddriven tag recommend abstract collabor tag user assign arbitrari keyword tag describ content object facilit navig improv search depend pre configur categori larg scale tagbas system tag recommend servic assist user assign tag object consolid vocabulari tag user promis approach tag recommend exploit occurr tag method challeng huge size tag vocabulari comput complex increas exponenti number tag score tag distort tag oper scale score directli compar paper propos method recommend tag demand driven basi initi set tag appli object reduc space solut complex increas polynomi size tag vocabulari score tag calibr entropi minim approach correct distort precis recommend conduct systemat evalu propos method type media audio bookmark video experiment result show propos method fast boost recommend qualiti experiment scenario instanc case popular audio site improv precis rang depend number tag input outperform recent propos occurr base tag recommend method activ learn genet program record dedupl abstract great major genet program gp algorithm deal classif problem follow supervis approach fit case evalu model label applic domain lot human effort requir label train data method semi supervis approach significantli reduc time requir data label maintain accept accuraci rate paper present activ learn gp agp semi supervis gp instanti data dedupl problem agp activ learn approach committe multi attribut function vote classifi record pair duplic committe major vote predict class data pair user call solv conflict method appli dataset compar dedupl method result show agp guarante qualiti dedupl reduc number label exampl need ondux ddemand unsupervis learn inform extract dsupervis approach extract attribut dvalu pair wikipedia articl explor featur automat identif user goal web search classifi document link dbase bibliometr measur model web hypergraph comput page reput structur inform improv search web collect editori call amp calendareditori statist featur find phrasal term text collect probabilist approach automat fill form dbase web interfac evid qualiti textual featur web automat fill form dbase web interfac free text input index web paper present googl prototyp larg scale search engin make heavi structur present hypertext googl design crawl index web effici produc satisfi search result exist system prototyp full text hyperlink databas million page http googl stanford engin search engin challeng task search engin index ten hundr million web page involv compar number distinct term answer ten million queri dai import larg scale search engin web academ research due rapid advanc technolog web prolifer creat web search engin todai year ago paper depth descript larg scale web search engin detail public descript date problem scale tradit search techniqu data magnitud technic challeng involv addit inform present hypertext produc search result paper address question build practic larg scale system exploit addit inform present hypertext problem effect deal uncontrol hypertext collect publish invert file research invert file compress focus compress ratio small index compress ratio import fast interact search read smaller index faster search text index techniqu paper present recent work multimedia retriev project cambridg univers olivetti research limit orl present techniqu treme rapid audio index rate approach thousand time real time unlik method techniqu depend fix vocabulari recognit system keyword advanc statist method develop text index techniqu rapid effici retriev brows audio video document paper present project background index retriev techniqu video mail retriev applic incorpor content base audio index retriev brows evolutionari approach combin sourc evid search engin strategi allow meaning compar score approxim match flexibl approach extract metadata bibliograph citat find templat web collect research develop web inform retriev brazilian univers local dbase prune method web search structur dbase crawl hidden web comput block import search web site strategi allow meaning compar score approxim match flux dcim flexibl unsupervis extract citat metadata hypergraph model comput page reput web collect abstract work propos represent web direct hypergraph graph link connect pair page pair disjoint set page model web hypergraph deriv web graph divid set page overlap block link page distinct block creat hyperarc hyperarc connect block page singl page creat goal provid reliabl inform link analysi method hypergraph structur comput reput web page experi hypergraph version previous propos link analysi method pagerank indegre present experi hypergraph version pagerank indegre produc result compar origin graph version approach xml path match cost deffect method detect web site replica search engin databas fast robust method web page templat detect remov compar studi citat link document classif link import sourc inform deal web collect question remain techniqu web appli collect document citat scientif paper work present compar studi digit librari citat web link context automat text classif show fact differ citat link context comparison run seri experi digit librari comput scienc paper web directori refer collect measur base citat tend perform page web directori gain text base classifi measur base bibliograph coupl perform digit librari propos simpl effect combin tradit text base classifi citat link base classifi combin base notion classifi reliabl present gain micro averag web collect signific gain obtain digit librari final user studi perform investig result discov misclassif citat link base classifi fact difficult case hard classifi human extract search inform web faq yaqcx word dbase queri dawar compressor xml data abstract xml de facto standard data exchang internet effici store queri xml data isstil open problem paper present yaqcx anotherqueri awar compressor xml yaqcx adopt word base model combin byte code provid effici approach compress decompress queri xml data implement subset xpath apow pattern match extens regular express rang queri partial match addition process queri access actual compress data asfewasposs solv predic content show result base experi show yaqcx compress ratio compar xmill close queri awar compressor xqzip xgrind wealso show yaqcx compress decompress faster xmill outperform xgrind queri process structur ddriven crawler gener gogetit tool gener structur ddriven web crawler site level nois remov search engin boom search engin industri determin onlin organ attempt artifici increas rank order attract visitor web site time growth web inher gener navig hyperlink structur neg impact import measur emploi current search engin paper propos evalu algorithm identifi noisi link web graph spam simpl relationship real world entiti repres site replic content unlik prior work target type noisi link structur resid site level page level investig annihil site level mutual reinforc relationship abnorm support come site complex link allianc web site experi link databas todobr search engin show strong increas qualiti output rank appli techniqu link dbase similar measur classif web document snif simpl nude imag finder detec de plica utilizando cont fado estrutura imped coupl content dtarget advertis applic cluster techniqu multipl sequenc align improv web search effici local base static prune method unargu fast continu growth volum index index document web pose great challeng search engin true search ectiv time space cienci paper present index prune techniqu target search engin address issu disconsid ect adopt prune strategi capabl greatli reduc size search engin indic experi real search engin show techniqu reduc indic storag cost tradit lossless compress method keep loss retriev precis minimum compar indic size compress compress rate higher eighth origin size importantli result due reduct storag overhead queri process time reduc origin time loss averag precis method yield signif improv compar static prune method search engin indic addit techniqu orthogon underli search algorithm adopt virtual search engin measur similar collect valu paper propos set similar metric manipul collect valu occur xml document data model present tax algebra treat xml element label order root tree xml node atom singl valu short charact string date complex nest structur node propos type similar metric mav atom node mcv complex node case suggest applic domain depend metric case defin metric complex valu structur depend andcanb distinctli appli tupl collect valu present experi show effect method discov search engin relat queri associ rule andar pampulha belo horizont brazil string process inform retriev abstract sequenc sn integ smaller polylog show repres nh bit sq answer rank select queri constant time order empir entropi nh inform theoret lower bound bit storag sequenc fix encod symbol extend previou result binari sequenc improv previou result gener sequenc queri answer log time larger repres nh log bit answer queri log log log time contribut paper show combin compress represent integ sequenc exist compress boost techniqu design compress full text index scale size input alphabet design variant fm index index string nhk bit storag hk order empir entropi space bound hold simultan log constant lt lt polylog index count occurr arbitrari pattern substr time locat pattern occurr log time constant lt lt report text substr length log time combin link dbase content dbase method web document classif associ rule discov search engin relat queri work present method automat gener suggest relat queri submit web search engin method extract inform log past submit queri search engin algorithm mine associ rule experiment result perform log million queri submit commerci search engin give correct suggest top suggest present common queri extract real log link inform similar measur web classif abstract object paper studi link structur web deriv similar measur document evalu measur determin accur predict subject web page experi web directori link extern page greatli increas qualiti result gain high point obtain compar text base classifi similar measur test work citat present perform determin web page relat work import insight similar measur deriv link appli web ir problem local versu global link inform web ad compress dgener text retriev system expan de consulta utilizando indexa sem ntica latent rank dpreserv dlevel cach scalabl search engin constru eficient de cdndice para base de dado textuai link dbase content dbase evidenti inform belief network model fast flexibl word search compress text text search complex approxim pattern algorithm time faster search uncompress text discuss impact techniqu invert file point logic block argu possibl keep text compress time decompress displai purpos categori subject descriptor code inform theori data compact work partial support siam project grant mct finep pronex amyri cyte project cape scholarship de moura fondecyt grant navarro baeza yate cnpq grant ziviani author address de moura ziviani dept comput scienc univ feder de mina gerai av antonio carlo belo horizont brazil navarro baeza yate dept comput scienc univ de chile av blanco encalada santiago chile permiss make digit compress kei dgener text retriev system articl discuss recent method compress text index text retriev system compress complet text index total amount space half size origin text surprisingli time requir build index answer queri index text compress case space time trade text compress time allow updat occur compress text ad compress block address invert index invert index compress block address sequenti search compress text techniqu separ develop effici low overhead text retriev modern text compress techniqu reduc text size search directli faster uncompress text invert index compress obtain signific reduct origin size process speed block address make invert list point text block exact posit pai reduct space sequenti text scan work combin idea singl scheme present compress invert file index compress text block address techniqu compress index studi perform respect block size compar index separ techniqu vari block size show index superior isol approach instanc extra space overhead index scan text exact search allow error match keyword text compress invert file block address text databas effici distribut algorithm build invert file introduct potenti larg size full text collect demand special index techniqu effici retriev exampl index suffix arrai invert file signatur file strong weak point invert file tradition popular index techniqu year invert file search strategi base vocabulari fit main memori invert file simpl updat perform pattern search form conjunct disjunct simpl word common type queri ir system web simpl structur task build invert file larg text collect web costli faster index algorithm desir parallel distribut hardwar gener index obviou altern regard hardwar platform network workstat present perform compar typic parallel machin cost effect effici distribut algorithm build invert file introduct potenti larg size full text collect demand special index techniqu effici retriev exampl index suffix arrai invert file signatur file strong weak point invert file tradition popular index techniqu year invert file search strategi base vocabulari fit main memori invert file simpl updat perform pattern search form conjunct disjunct simpl word common type queri ir system web simpl structur task build invert file larg text collect web costli faster index algorithm desir parallel distribut hardwar gener index obviou altern regard hardwar platform network workstat present perform compar typic parallel machin cost effect linear time sort skew distribut fast search compress text allow error present fast compress decompress scheme natur languag text effici flexibl string match search compress text directli compress scheme word base huffman encod code alphabet byte orient bit orient compress typic english text origin size compress gzip compress time close time compress approxim half time gzip decompress time lower gzip compress search algorithm larg number variat exact approxim compress string match problem phrase rang complement wild card arbitrari regular express separ stopword discard search time significantli increas cost algorithm base word orient shift algorithm fast boyer moor type filter concomitantli vocabulari text part huffman code data search simpl pattern experi show run algorithm compress text fast run agrep uncompress version text search complex approxim pattern algorithm time faster agrep mention impact techniqu invert file point document logic block glimps direct pattern match compress text present fast compress decompress techniqu natur languag text novelti exact search compress text directli sequenti pattern match algorithm approxim search effici decod compress scheme semi static word base model huffman code code alphabet byte orient bit orient bit byte mark begin word search compress pattern directli compress text achiev compress ratio typic english text search simpl pattern experi show run algorithm compress text fast run agrep uncompress version text search complex approxim pattern algorithm time faster agrep 